Home monitoring method and apparatus

ABSTRACT

Provided is a home monitoring method and apparatus. A home gateway executes receiving, from a mobile terminal, a request message for requesting home monitoring, providing, to the mobile terminal in response to the request message, monitoring information associated with a camera module included in at least one home device registered in advance and/or a position to be monitored, receiving, from the mobile terminal, selection information indicating a first camera module selected based on the monitoring information, transmitting, to the mobile terminal, image data captured and collected by the first camera module in response to the reception of the selection information, receiving, from the mobile terminal, a control command with respect to the first camera module, and transmitting the control command to a first home device including the first camera module.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/262,174, filed on Jan. 30, 2019, which is a continuationapplication of prior application Ser. No. 14/105,690, filed on Dec. 13,2013, which was based on and claimed priority under 35 U.S.0 § 119(a) ofa Korean patent application number 10-2012-0146251, filed on Dec. 14,2012, in the Korean Intellectual Property Office, and of a Korean patentapplication number 10-2013-0106518, filed on Sep. 5, 2013, in the KoreanIntellectual Property Office, the disclosure of each of which isincorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a home network system, andparticularly, to a home monitoring method and apparatus using electricalappliances and cameras.

2. Description of the Related Art

A home network system refers to a system that connects home devicesinstalled in a house through a wired or wireless network so as tocontrol the home devices. An advanced home network system integrateshome devices through a Home Gateway (HGW or H-GW) and connects the homedevices to an external common data network, for example, an InternetProtocol (IP) network (that is, the Internet), so as to provide morevarious Internet-based services. The home network system may controlhome devices based on a request from a user and may provide a servicedesired by the user.

Recently, types of home devices have gradually diversified, andaccordingly, user's requests have diversified and thus, there is a needto provide the best service by taking into consideration variousrequests from a user.

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

Accordingly, an aspect of the present disclosure is to provide a methodand apparatus for providing a service for a home network system.

Another aspect of the present disclosure is to provide a home monitoringmethod and apparatus through a home network system.

Another aspect of the present disclosure is to provide a method andapparatus for monitoring circumstances in a house through an electricalappliance and an IP camera in a home network system.

Means for Solving Poblems

In accordance with another aspect of the present disclosure, there isprovided a home monitoring method by a home gateway included in a homenetwork system, the method including: receiving, from a mobile terminal,a request message for requesting home monitoring; providing, to themobile terminal in response to the request message, monitoringinformation associated with a camera module included in at least onehome device registered in advance and/or a position to be monitored;receiving, from the mobile terminal, selection information indicating afirst camera module selected based on the monitoring information;transmitting, to the mobile terminal, image data captured and collectedby the first camera module in response to the reception of the selectioninformation; receiving, from the mobile terminal, a control command withrespect to the first camera module; and transmitting the control commandto a first home device including the first camera module.

In accordance with another aspect of the present disclosure, there isprovided a home monitoring method by a mobile terminal that isregistered in a home gateway included in a home network system, themethod including: transmitting a request message for requesting homemonitoring to the home gateway; receiving, from the home gateway,monitoring information associated with a camera module included in atleast one home device registered in advance and/or a position to bemonitored, in response to the transmission of the request message;transmitting, to the home gateway, selection information indicating afirst camera module selected based on the monitoring information;receiving, from the home gateway, image data captured and collected bythe first camera module, in response to the transmission of theselection information; and transmitting, to the home gateway, a controlcommand input by a user with respect to the first camera module.

In accordance with another aspect of the present disclosure, there isprovided a home gateway apparatus that supports home monitoring in ahome network system, the apparatus including: a network interface modulethat receives a request message for requesting home monitoring from amobile terminal through wireless and wired networks, provides, to themobile terminal, monitoring information associated with a camera moduleincluded in at least one home device registered in advance and/or aposition to be monitored, in response to the request message, receives,from the mobile terminal, selection information indicating a firstcamera module selected based on the monitoring information, transmits,to the mobile terminal, image data captured and collected from the firstcamera module in response to the reception of the selection information,and receives a control command with respect to the first camera modulefrom the mobile terminal; and at least one communication module that isconfigured to enable communication with a camera module included in atleast one home device registered in the home network system, and thattransmits the control command to a first home device equipped with thefirst camera module.

In accordance with another aspect of the present disclosure, there isprovided a mobile terminal apparatus that executes home monitoringthrough a home gateway included in a home network system, the mobileterminal apparatus including: a mobile communication module thattransmits a request message for requesting home monitoring to the homegateway through wireless and wired networks, receives, from the homegateway, monitoring information associated with a camera module includedin at least one home device registered in advance and/or a position tobe monitored, in response to the transmission of the request message,transmits, to the home gateway, selection information indicating a firstcamera module selected based on the monitoring information, receives,from the home gateway, image data captured and collected by the firstcamera module, in response to the transmission of the selectioninformation, and transmits, to the home gateway, a control command inputby a user with respect to the first camera module; and a display modulethat displays the image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a simplified structure of a home network systemaccording to an embodiment of the present disclosure;

FIG. 2 illustrates a simplified configuration of a home network systemthat is capable of connecting with a mobile terminal according to anembodiment of the present disclosure;

FIG. 3 is a block diagram illustrating a schematic configuration of ahome gateway according to an embodiment of the present disclosure;

FIG. 4 is a block diagram illustrating a schematic configuration of amobile terminal according to an embodiment of the present disclosure;

FIG. 5 illustrates a schematic configuration of a home network systemthat supports home monitoring according to an embodiment of the presentdisclosure;

FIG. 6 illustrates a configuration of a mobile home device that isequipped with a camera according to an embodiment of the presentdisclosure;

FIG. 7 is a message flowchart illustrating a registration procedure of ahome device according to an embodiment of the present disclosure;

FIG. 8 is a message flowchart illustrating a registration procedure of amobile terminal according to an embodiment of the present disclosure;

FIG. 9 illustrates a home monitoring procedure that is performed in ahome network system according to an embodiment of the presentdisclosure;

FIG. 10 illustrates a home monitoring procedure that is performed in ahome network system according to another embodiment of the presentdisclosure;

FIG. 11 is a message flowchart illustrating a home monitoring procedureaccording to an embodiment of the present disclosure;

FIG. 12 is a message flowchart illustrating a home monitoring procedureaccording to another embodiment of the present disclosure;

FIG. 13 is a message flowchart illustrating a home monitoring procedureaccording to another embodiment of the present disclosure;

FIG. 14 illustrates a User Interface (UI) for executing a remotecontrolling application in a mobile terminal for home monitoringaccording to an embodiment of the present disclosure;

FIG. 15 illustrates a configuration of a screen for home monitoring of aremote controlling application according to an embodiment of the presentdisclosure;

FIG. 16 illustrates a configuration of a screen for home monitoring of aremote controlling application according to another embodiment of thepresent disclosure;

FIG. 17 illustrates a display of image data according to an embodimentof the present disclosure;

FIG. 18 illustrates a display of image data according to anotherembodiment of the present disclosure;

FIGS. 19A and 19B are screens illustrating a control of a mobile cameraaccording to an embodiment of the present disclosure;

FIG. 20 illustrates an alarm for an abnormal circumstance associatedwith a home network system in a mobile terminal according to anembodiment of the present disclosure;

FIG. 21 illustrates a configuration of a screen for home monitoring of aremote controlling application according to another embodiment of thepresent disclosure;

FIG. 22 illustrates a display of an abnormal circumstance associatedwith a home network system in a mobile terminal according to anotherembodiment of the present disclosure; and

FIG. 23 illustrates calculation of a movement vector of a mobile homedevice according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Hereinafter, various embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings.Further, in the following description of the present disclosure, adetailed description of known functions and configurations incorporatedherein will be omitted when it may make the subject matter of thepresent disclosure rather unclear. Then, terms described later aredefined in consideration of the functions of the present disclosure, butmay vary according to the intention or convention of a user or operator.Therefore, the definition needs to be determined based on the overallcontents of the present specifications.

FIG. 1 illustrates a simplified structure of a home network systemaccording to an embodiment of the present disclosure.

Referring to FIG. 1, a home network system is configured to include homedevices 110 having a control and communication function and a HomeGateWay (HGW) 120. The home devices 110 are located inside (or outside)a house, and include smart appliances 112, security devices 114,lighting devices 116, energy devices 118, and the like. For example, thesmart appliances 112 correspond to a Television (TV), anair-conditioner, a refrigerator, a washing machine, a robot-cleaner, ahumidifier, and the like. The security devices 114 correspond to adoor-lock, a security camera, a Closed Circuit Television (CCTV), and asecurity sensor that senses a contact, a sound, a movement, and thelike, the lighting devices 116 correspond to a Light Emitting Diode(LED), a lamp, and the like, and the energy devices 118 may correspondto a heater, a power meter, a power socket, an electrical outlet, amultiple-tap, and the like. In addition, the home devices 110 mayinclude a Personal Computer (PC), an IP camera, an Internet phone, awired/wireless phone, a mobile phone used in a home, a curtain or blindthat may be electrically controlled, and the like.

The home devices 110 may be capable of performing communication with thehome gateway 110 according to a wired or wireless communication scheme,and may be configured to receive a control command from the home gateway120, to operate based on the control command, and to transmit requestedinformation and/or data to the home gateway 120.

The home gateway 120 may be embodied as an independent apparatus, or anapparatus equipped with a home gateway function. For example, the homegateway 120 may be embodied as a television, a cellular phone, a tabletcomputer, a set-top box, a robot cleaner, or a personal computer. Thehome gateway 120 may be equipped with corresponding communicationmodules for performing communication with the home devices 110 based ona wired or wireless communication scheme, so as to register and storeinformation associated with the home devices 110, to manage and controloperations, supportable functions, and states of the home devices 110,and to collect and store required information from the home devices 110.Particularly, the home gateway 120 may be connected with a data networksuch as the Internet, that is, an IP network 130, and allows aconnection by a communication terminal through the Internet 130 andtransfers a control signal received from the communication terminal to acorresponding home device. Also, the home gateway 120 may communicatewith the communication terminal using a wireless communication schemesuch as WiFi (Wireless Fidelity), Zigbee, Bluetooth, an NFC (Near FieldCommunication), and z-wave.

The home network system configured as described above may provide a homeentertainment service, such as an Internet TV (IPTV) through theInternet 130, a Video on Demand (VoD) and the like, a home datacommunication service such as data sharing, Voice over IP (VoIP), avideo phone, and the like, and a home automation service such as remotecontrol of an electrical appliance, a remote meter reading, crimeprevention, disaster prevention and the like. That is, the home networksystem may connect all types of home devices used inside and outside ahouse through a single network for a control.

Meanwhile, a user may connect with a home gateway included in the homenetwork system from the outside of the house using a communicationterminal such as a mobile terminal, or may remotely connect with eachhome device through the home gateway. The mobile terminal may be, forexample, a Personal Digital Assistant (PDA) including a communicationfunction, a Smart Phone, a feature phone, a tablet Personal Computer(PC), a notebook and the like, and may access a home network systemdirectly or through a provider's network and the Internet.

FIG. 2 illustrates a simplified configuration of a home network systemthat is capable of connecting with a mobile terminal according to anembodiment of the present disclosure.

Referring to FIG. 2, the home devices 110 in the home network systemconnects with the home gateway 120 through a wired or wirelesscommunication scheme. The home gateway 120 may connect with a wirednetwork 230 such as the Internet. A registered mobile terminal 250 mayconnect with a wireless network 240 including a wireless access networkand a provider core network, and may access the home gateway 220 throughthe wired network 230. A wireless network 240 may be a 2G(Generation) or3G cellular communication system, 3GPP(3rd Generation PartnershipProject), a 4G communication system, LTE (Long-Term Evolution), WiMAX(World Interoperability for Microwave Access), and the like. Also, themobile terminal 250 may directly communicate with the home gateway 220using a wireless communication scheme such as WiFi, Zigbee, Bluetooth,an NFC, and z-wave.

FIG. 3 is a block diagram illustrating a schematic configuration of ahome gateway according to an embodiment of the present disclosure. Ahome gateway may be configured to include at least one of theillustrated component elements.

Referring to FIG. 3, a home gateway 300 may be connected with homedevices (not illustrated) using at least one communication module 312through 314. The communication modules 312 through 314 include at leastone of a wireless LAN module 312 such as WiFi, a Zigbee module 314, aBluetooth module 316, an NFC module 318, and a wired communicationmodule 320. Also, the home gateway 300 includes a controller 310, anetwork interface module 322, a User Interface (UI) 324, and a storageunit 326.

The controller 310 includes a CPU (Central Processing Unit), a Read-OnlyMemory (ROM) storing a control program for controlling the home gateway300, and a Random Access Memory (RAM) used as a storage area for worksperformed in the home gateway 300. The controller 310 may communicatewith home devices through the communication modules 312 through 320 byexecuting programs stored in the ROM or RAM or application programs thatmay be stored in the storage unit 326, and may generate and transmit acontrol command to the home devices or may store information collectedfrom the home devices in the storage unit 326.

The user interface 324 includes an output module such as a display, aspeaker, and a notification lamp, and an input module such as a touchscreen, a keypad, and a microphone, and may be used when a user directlycontrols the home gateway 300, registers or deletes home devices in/fromthe home gateway 300, or controls home devices through the home gateway300.

The network interface 322 may be, for example, an Internet communicationmodule, and may connect the home gateway 300 to an external network.

The storage unit 326 is configured to store a program code, data, orinformation required for an operation of the home gateway 300, under acontrol of the controller 310, and may store a high capacity datatransferred from external devices or home devices when needed. Also, thestorage unit 326 may further store at least one of additionalinformation required for controlling a home network system, for example,an arrangement plan (or a map) including an arrangement of sectionsincluded in a house (such as, a room, a kitchen, a living room, abathroom, and the like), information associated with home devicesincluded in a home network system, information associated with at leastone target position to be secured and monitored, and movement commandsfor controlling a movement of a mobile home device.

FIG. 4 is a block diagram illustrating a schematic configuration of amobile terminal according to an embodiment of the present disclosure. Amobile terminal may be configured to include at least one of theillustrated component elements.

Referring to FIG. 4, a mobile terminal 400 is configured to include acontroller 410, a cellular mobile communication module 420, asub-communication module 422, a multimedia module 428, a camera module436, a GPS (Global Positioning System) module 438, an input/outputmodule 444, a sensor module 442, a storage unit 458, a power supplier440, and at least one display module 460. The sub-communication module130 includes at least one of a wireless LAN module 424 and ashort-distance communication module 426, and the multimedia module 428includes at least one of a broadcasting communication module 430, anaudio playback module 432, and a video playback module 434. The cameramodule 436 includes at least one camera, and the input/output module 444includes at least one of a button set 446, a microphone 448, a speaker450, a vibration motor 452, a connector 454, and a keypad 456.

The controller 410 includes a CPU, a ROM storing a control program forcontrolling the mobile terminal 400, and a RAM used as a storage areafor storing a signal or data input from the outside of the mobileterminal 400 or for storing works performed in the mobile terminal 400,and may control other component elements of the mobile terminal 400.

The cellular mobile communication module 420 connects the mobileterminal 400 to an external apparatus (particularly, a base station of acellular system) through at least one or a plurality of antennas (notillustrated), using a wireless access technology based on a cellularcommunication protocol according to a control of the controller 410. Thecellular mobile communication module 420 transmits/receives a wirelesssignal including a voice call, a video call, a Short Messaging Service(SMS) message, or a Multimedia Messaging Service (MMS) message to/fromanother apparatus that is capable of performing communication.

The wireless LAN module 424 may connect with the Internet according to acontrol of the controller 410 in a place where a wireless AP (accesspoint) (not illustrated) is installed. The short-distance communicationmodule 426 may execute a wireless short-distance communication betweenthe mobile terminal 400 and an external apparatus according to a controlof the controller 410. A short-distance communication scheme may includeBluetooth, Zigbee, IrDA (infrared data association) and the like.

The display 460 displays information associated with variousapplications (for example, calling, data transmission, broadcasting, acamera, and the like) that may be executed by the controller 410, andmay be configured as a touch screen that provides a user interface thatis configured to be adjusted to the same. The controller 410 may enablea soft key displayed on the touch screen 460 to be selected or mayexecute an application or a function corresponding to the soft key, inresponse to a user gesture sensed on the touch screen. The user gestureincludes a touch by a finger or an instrument, motion recognition by abody part, and the like.

A user located in the outside may desire a home network systemconfigured as described above to execute a monitoring function withrespect to the circumstances in a house. In this case, the user mayconnect with a home gateway using a mobile terminal or a computingapparatus, and may monitor the circumstances in the house by controllingat least one of the registered home devices through the home gateway.

FIG. 5 illustrates a schematic configuration of a home network systemthat supports home monitoring according to an embodiment of the presentdisclosure.

Referring to FIG. 5, a home network system 500 includes a home gateway510, at least one stationary home device 512 a and 512 b (commonlycalled 512) equipped with a camera module (or a stationary camera), andat least one mobile home device equipped with a camera module (or amobile camera) 514. The stationary home device 512 may be a ClosedCircuit Television (CCTV) installed in a predetermined location such asa wall or a ceiling, an electrical appliance equipped with a cameramodule such as an air-conditioner or refrigerator, and the like, and maybe connected with the home gateway 510 through a wired interface or maybe connected with the home gateway 510 through a wireless interface.

The mobile home device 514 may be a robot cleaner, a mobile toy, or thelike that includes a movement function, and may be connected with thehome gateway 510 through a wireless interface and moves based on acontrol of the home gateway 510. At least one of the home devices 512and 514 includes a function of sensing a movement through a camera or asensor, and may report a movement sensing result to the home gateway510. Also, at least one of the home devices 512 and 514 may be equippedwith a light such as an LED so as to capture an image in a dark room.

In addition, one or more sensing modules 516 that are installed in aposition that requires monitoring may be connected with the home gateway510 through a wired or wireless interface. The sensing module 516 may bea door sensor that senses intrusion through a front door, a magneticsensor that senses breaking into a window, a ultrasonic sensor, a gassensor that senses a leakage of gas from a gas range or the like, a heatsensor that senses a fire, a movement sensor that senses a movement ofan intruder or a pet, a microphone that senses an abnormal sound, andthe like, and may include a signal transmitting module that monitorsoccurrence of various abnormal circumstances, and wiredly or wirelesslytransmits an abnormal signal including corresponding identificationinformation when a corresponding abnormal circumstance is sensed. Forexample, the sensing module 516 may be installed in a fixed positionthat requires monitoring or may be included in a mobile home device. Theabnormal signal may be collected by the home gateway 510 or may bedirectly transferred to the home devices 512 and 514 equipped with acamera.

The home gateway 510 registers information associated with the homedevices 512 and 514, and manages states. When providing the stationaryhome device 512, the home gateway 510 may register and manage a positionwhere the home device 512 is installed, for example, a first CCTV in amain room, and second and third CCTVs in a living room. When providingthe mobile home device 514, the home gateway 510 registers and manages adocking position of the home device 514 (for example, a charger of arobot cleaner), and additionally senses and traces a current position ofthe home device 514 using an RF (Radio Frequency) tag, a sensor, acamera included in the home device 514, and the like.

Also, the home gateway 510 may transmit, to the home device 514, amovement command for controlling a movement, and may store a history ofthe movement command together with time information. For example, thehome gateway 510 stores movement commands that instruct the home device514 to move from a docking position to at least one position to bemonitored, and to return to the docking position again. The at least oneposition to be monitored may be, for example, a place where a stationarysensing module is not installed, a place that is vulnerable to security,or a place where a child or a pet exists, and may be set by a user. Asanother example, the home gateway 510 may set the home device 514 topatrol a path between a docking position and a position (or positions)to be monitored when the home network system is operated in a securitymode.

The home gateway 510 may collect image data captured through camerasincluded in the home devices 512 and 514. Collecting the image data maybe continuously executed in real time or may be begun based on apredetermined event condition. The image data may be stored in the homegateway 510 or an external storage apparatus during a predeterminedperiod of time or until it is deleted by a user.

A user may connect with the home gateway 510 through the Internet 520using a personal computer 522 in an office, and may check image datacollected by the home gateway 510. Also, the user may connect with thehome gateway 510 through a wireless network 530 and the Internet 520using a mobile terminal 532 in a mobile environment, and may determineimage data collected by the home gateway 510.

When directions of cameras included in the stationary home devices 512are adjustable, a user transmits a control command to the home gateway510 through the personal computer 522 or the mobile terminal 532, andthe home gateway 510 transfers the control command to a correspondinghome device 512. The control command may include at least one of anupper/lower/left/right side adjusting command for adjusting directionsof stationary cameras and a zooming-in/out (Zoom In/Out) command. Then,the home device 512 controls a camera lens in response to the controlcommand, and transfers image data captured through the controlled cameralens to the home gateway 510.

Also, the user transmits, to the home gateway 510, a control command forcontrolling a movement of a mobile camera (or a mobile home device 514equipped with a camera module) 514 through the personal computer 522 orthe mobile terminal 532, and the home gateway 510 transfers the controlcommand to a corresponding home device 514. The control command includesa left/right rotation command, a forward/backward movement command,and/or a zooming in/out command for a movement of the home device 514.Then, the home device 514 controls or moves a camera lens in response tothe control command, and transfers image data captured through thecontrolled camera lens to the home gateway 510.

FIG. 6 illustrates a configuration of a mobile home device that isequipped with a camera according to an embodiment of the presentdisclosure. Here, a configuration of a home device embodied as a robotcleaner is illustrated. The robot cleaner may be configured to includeat least one of the illustrated component elements.

Referring to FIG. 6, a robot cleaner 600 is configured to include acontroller 610, a camera module 612, a user interface 614, a sensormodule 616, a communication module 618, a storage unit 620, a cleaningmotor 622, a movement motor 624, a direction motor 626, and a powersupplier 628.

The controller 610 includes a CPU, a ROM storing a control program forcontrolling a cleaning function and a movement of the robot cleaner 600,and a RAM used as a storage area for storing a signal or data input fromthe outside of the robot cleaner 600 or for storing works performed inthe robot cleaner 600, may control a function of the robot cleaner 600based on a stored control program or a control command received from theoutside, and may control the communication module 618 to transmit imagedata collected by the camera module 612 to a designated destination.

The camera module 612 is fixedly mounted on a predetermined position onan outer side of the robot cleaner 600, and provides image datagenerated by capturing an ambient image. The camera module 612 may beconfigured to mechanically adjust a direction of a photography lens, andmay adjust a direction of the photography lens to anupper/lower/left/right side by a predetermined degree, or may executezooming in/out, in response to the control command from the controller610.

The user interface 614 is configured to include a key input unit to seton/off of a power of the robot cleaner 600, an unique function of therobot (for example, a cleaning function and the like), and a displayunit indicating a state, a function, a communication state, and the likeof the robot cleaner 600. The sensor module 616 is configured to includea heat sensor, a motion sensor, or the like, and reports, to thecontroller 610, that a motion is sensed when the motion is sensed withina detectable range. The communication module 618 is configured toinclude a mobile communication module and/or a wireless LANcommunication module, and may transmit and receive a wireless signalto/from a home gateway and/or an external apparatus. In particular, thecommunication module 618 receives a control command from a home gatewaythrough a wireless LAN or Zigbee, and may transmit image data collectedby the camera module 612 to the home gateway through a wireless LAN or amobile communication scheme.

The storage unit 620 may store image data captured by the camera module612 under a control of the controller 610 during a predetermined periodof time or up to a predetermined capacity, and may temporarily storedata processed for image monitoring by the robot cleaner 600. Also, thestorage unit 620 may further include at least one of additionalinformation required for a movement of the robot cleaner 600, forexample, an arrangement plan including an arrangement of a room, akitchen, a living room, and a bathroom in a house, informationassociated with a docking position, for example, a charging system,information associated with at least one target position to bemonitored, and movement commands for a movement between the dockingposition and the target position.

The cleaning motor 622 is operated based on a control of the controller610 and executes a cleaning function of the robot cleaner 600. Themovement motor 624 is operated based on a control of the controller 610,so as to enable the robot cleaner 600 to move forward or backward. Thedirection motor 626 is operated based on a control of the controller610, so as to enable the robot cleaner 600 to change a direction of amovement. The power supplier 628 supplies a voltage stored in arechargeable battery to each operation circuit required for driving therobot cleaner 600, and charges the rechargeable battery with powersupplied from the outside.

In the home network system configured as shown in FIG. 5, a home gatewayregisters information associated with home devices installedinside/outside a house, and controls and manages the registered homedevices. For example, when a new home device is installed, the homegateway collects and stores at least one of information associated withthe home device, for example, an identifier of the home device,positional information, capacity information, and state information. Theinformation associated with the home device may be directly obtainedfrom the home device through a service discovery procedure between thehome gateway and the home device, may be received by the home gatewayfrom a server on the Internet based on identification informationprovided from the home device, or may be directly input to the homegateway by a user or a system administrator. For example, when a userpurchases and installs a new home device, and inputs, into the homegateway, identification information provided through instructions, forexample, a serial number, a model number, a Quick Response (QR) code,the home gateway may connect with a server of a manufacturer on theInternet based on the identification information so as to obtain theinformation associated with the home device.

FIG. 7 is a message flowchart illustrating a registration procedure of ahome device according to an embodiment of the present disclosure.

Referring to FIG. 7, a home device is installed and power is suppliedfirst in step 702. Alternatively, a user determines to register apreviously installed home device in a home gateway. In step 704, thehome gateway executes a search procedure for a home device. For example,when providing that the home device is connected with the home gatewaythrough WiFi, the home gateway executes a search procedure for a newdevice around the home gateway, periodically or in response to a requestfrom a user. When a home device is detected by the home gateway throughthe search procedure, the home gateway may receive informationassociated with the home device from the home device. The informationassociated with the home device includes at least one of an identifierof the home device, capacity information, and state information. Forexample, when the home device is a security camera, the capacityinformation includes whether an angle of the security camera isadjustable, whether the security camera supports zooming in/out, and thelike. The state information includes a position where the securitycamera is installed.

As an optional embodiment, in step 706, the home gateway receives aregistration request associated with a home device to be registered,directly from a user through an its own user interface, a controlconsole connected by an external interface, or a computer on theInternet, and receives an input of information associated with the homedevice from the user. As another embodiment, the home gateway may obtainidentification information associated with a home device and/or stateinformation from the home device or a user, and may obtain capacityinformation associated with the home device by connecting with a serverof a manufacturer on the Internet based on the identificationinformation.

In step 708, the home gateway stores the information associated with thehome device, and manages the home device as a registered home device. Asan optional embodiment, the home gateway may receive an input ofadditional information associated with the home device directly from auser or from a remote control terminal through the Internet or WiFi. Theadditional information may include at least one of, for example, a nameor a nickname of the home device, and a position where the home deviceis installed.

As another optional embodiment, the home gateway may receive an input ofadditional information associated with a mode and/or authority of thehome device directly from a user or from a remote control terminalthrough the Internet or WiFi. The additional information includes, forexample, a level of access right with respect to the home device. Thelevel of the access right includes, for example, a priority ofauthority, allowing only an access through a home gateway, allowing anaccess by a registered remote control terminal, allowing an access by aremote control terminal registered in the home gateway, or the like.

In step 710, the home gateway sets a connection with the home devicewhen needed. Setting of the connection may be executed based on aprocedure determined in a communication scheme between the home gatewayand the home device, that is, WiFi, Zigbee, Bluetooth, NFC, or a wiredcommunication scheme. As described above, when a connection between thehome gateway and the home device is set, in step 712, the home devicemay transmit data to the home gateway through the set connection whenneeded. As an embodiment, when the home device is a security camera oran electrical appliance equipped with a security camera function, thehome device transmits captured image data to the home gateway in realtime. Also, when needed, in step 714, the home gateway transmits acontrol command to the home device. As an example, the home gatewaytransmits an angle adjusting command and/or zooming in/out command for asecurity camera included in a home device, and the home device adjustsan angle of a camera lens or executes zooming in/out in response to thecommand, and continuously transmits image data captured in the adjustedcondition. As another embodiment, when the home device is anair-conditioner, the control command may include a desired temperature,a desired wind intensity, an operation time, a scheduled time, and thelike. According to a communication scheme that is applied, the homedevice may provide the home gateway with a result of execution of thecontrol command as a response.

In the home network system configured as shown in FIG. 5, a userregisters a Personal Computer (PC) or a mobile terminal (Mobile Station(MS)) for remote controlling in the home gateway, and the home gatewayrestrictively allows remote controlling by the registered PC or mobileterminal. As an example, the user may register, in the home gateway, amobile terminal to be used for remote controlling, manually or throughuse of a mobile communication network or WiFi, and the home gatewayreceives and stores information associated with the mobile terminal, forexample, an identifier of the mobile terminal, authority for remotecontrolling, and information associated with a user mode. Theinformation associated with the mobile terminal may be obtained directlyfrom the mobile terminal through a connection between the home gatewayand the mobile terminal, may be received by the home gateway from aserver of a wireless communication system based on identificationinformation provided from the mobile terminal, or may be input directlyto the home gateway from a user or a system administrator.

FIG. 8 is a message flowchart illustrating a registration procedure of amobile terminal according to an embodiment of the present disclosure.

Referring to FIG. 8, a mobile terminal installs a remote controllingapplication (hereinafter referred to as an app) that supportscontrolling of a home network system in step 802. As an optionalembodiment, installation of the remote controlling app may be limited toonly a mobile terminal of a user that passes a predeterminedauthentication procedure. As an example, the home gateway provides anauthentication code, and the home gateway, a server of a manufacturer ofthe home gateway, or a server for providing a remote controlling appallows downloading of the remote controlling app only to a mobileterminal of a user who inputs the authentication code. As anotheroptional embodiment, the remote controlling application mayrestrictively provide a function based on an authority level of a mobileterminal. As an example, the remote controlling app may selectivelyprovide, based on an authority level permitted by the home gateway, atleast one of the functions of inquiring about information through thehome gateway, receiving an alarm from the home gateway, partiallycontrolling home devices, and completely controlling home devices.

In step 804, the mobile terminal executes the remote controlling app,and transmits, to the home gateway, a registration request messagethrough the remote controlling app. The remote controlling app may betransferred to the home gateway through the Internet, or may be directlytransferred to the home gateway through WiFi. The registration requestmessage includes an identifier of the mobile terminal (for example, anMSID (MS Identifier) or IMSI (International Mobile SubscriberIdentify)), a login ID and password, authority information, and modeinformation. The authority information indicates a level of access rightwith respect to a home network system, for example, allowing inquiringabout information through the home gateway, allowing receiving of analarm from the home gateway, partially allowing controlling on homedevices, and completely allowing controlling on home devices.

As an optional embodiment, in step 806, the home gateway receives theregistration request associated with the mobile terminal to beregistered, from a user through an its own user interface, a controlconsole connected by an external interface, or a computer on theInternet, and receives an input of information associated with themobile terminal, for example, authority information and/or modeinformation from the user.

As an optional embodiment, in step 808, the home gateway connects with asubscriber server of a mobile communication network based onidentification information associated with the mobile terminal obtainedfrom the mobile terminal or a user, so as to authenticate the mobileterminal. When the authority information of the mobile terminal is notinput, the home gateway may generate authority information associatedwith the mobile terminal based on a result of the authentication.

In step 810, the home gateway stores the information associated with themobile terminal, and manages the mobile terminal as a registered mobileterminal. In step 812, the home gateway transmits, to the mobileterminal, a registration approval message indicating that registrationof the information associated with the mobile terminal succeeds.

In step 814, when needed, the home gateway may transmit, to the mobileterminal, data required for remote controlling. As an embodiment, thehome gateway transmits, to the mobile terminal, image data captured byat least one security camera installed inside/outside a house, in realtime, periodically, or in an event triggering manner. As anotherembodiment, the home gateway transmits, to the mobile terminal, stateinformation associated with a light in/around a house, powerconsumption, and an open/shut state of a door, in real time,periodically, or in an event triggering manner. In step 816, whenneeded, the mobile terminal transmits, to the home gateway, a controlcommand for the home gateway or at least one home device. As an example,the mobile terminal requests the home gateway to transmit image datacaptured by a security camera installed in a predetermined position inreal time. As another embodiment, the mobile terminal transmits an angleadjusting command and/or a zooming in/out command to a security camerathrough the home gateway.

FIG. 9 illustrates a home monitoring procedure that is performed in ahome network system according to an embodiment of the presentdisclosure.

Referring to FIG. 9, a home network system 900 includes a home gateway910, at least one stationary home device 912 a and 912 b (commonlycalled 912) equipped with a camera, and at least one mobile home device914 equipped with a camera. The stationary home device 912 may be a CCTVinstalled in a predetermined position such as a wall or a ceiling, anelectrical appliance equipped with a camera module such as anair-conditioner or a refrigerator, and the like, and may be connectedwith the home gateway 910 through a wired interface or may be connectedwith the home gateway 910 through a wireless interface. The mobile homedevice 914 may be a robot cleaner, a mobile toy, or the like thatincludes a movement function, and may be connected with the home gateway910 through a wireless interface.

The home gateway 910 may operate the home network system 900 in asecurity mode based on a request from a user or a predetermined setting.When the security mode is begun, the home device 912 and 914 equippedwith a camera begins capturing image data, and stores the image data orreports the image data to the home gateway 910. As an embodiment, whenthe security mode is begun, the home gateway 910 instructs the mobilehome device 914 to move to a predetermined position that is set inadvance, for example, a window in a kitchen, a back door, or the like,and to execute capturing image data and a sensing function, in responseto a request from a user. In particular, the mobile home device 914 maybe used for strengthening monitoring of a position where a stationarysensing module is not installed and is vulnerable to security.

When a user who is located outside the home network system 900 desiresto check circumstances inside/around a house, the user may executes aremote controlling app that supports home monitoring through a mobileterminal 922 and selects a home monitoring function. Then, in step 930,the mobile terminal 922 connects with the home gateway 910 directly orthrough a network 920, and transmits a request message for requestinghome monitoring to the home gateway 910. In step 932, the home gateway910 collects image data from at least one home device 912 and 914equipped with a camera, in response to the request message. In step 934,the home gateway 910 transmits, to the mobile terminal 922, the imagedata collected from the at least one home device 912 and 914. The mobileterminal 922 displays the image data so that a user checks the imagedata.

As an embodiment, the home gateway 910 collects the image data from allof the home devices 912 and 914 equipped with a camera in a system, andprovides the image data to the mobile terminal 922. As anotherembodiment, the home gateway 910 collects the image data from at leastone home device 912 and 914 registered in advance for home monitoring,and provides the image data to the mobile terminal 922.

The image data may be transferred in an original resolution or in a formof a compressed small resolution, based on a setting of a remotecontrolling app executed in the mobile terminal 922. When the homedevice 912 and 914 transmits image data to the home gateway 910 in realtime, and the home gateway 910 stores the image data, the home gateway910 transmits, to the mobile terminal 922, the image data collected inreal time without a request for transmission of the image data. Asanother embodiment, when the home gateway 910 receives a request messagefor requesting home monitoring from the mobile terminal 922, the homegateway 910 requests at least one home device 912 and 914 to transmitimage data and receives the image data from the home device 912 and 914in response to the request.

In step 936, the mobile terminal 922 transmits, to the home gateway 910,a control command for selecting an image of another camera, or foradjusting a position, an angle, a zoom of a predetermined camera. Instep 938, the home gateway 910 transfers the control command to a homedevice (for example, the home device 914) equipped with a selectedcamera. The home device 914 transfers, to the home gateway 910, imagedata captured after moving or controlling a camera lens in response tothe control command in step 940, and the home gateway 910 transfers theimage data to the mobile terminal 922 in step 942. The mobile terminal922 displays the image data so that the user checks the image data.

FIG. 10 illustrates a home monitoring procedure that is performed in ahome network system according to another embodiment of the presentdisclosure.

Referring to FIG. 10, a home network system 1000 includes a home gateway1010, at least one stationary home device 1012 equipped with a camera,and at least one mobile home device 1014 equipped with a camera. Thestationary home device 1012 may be a CCTV installed in a predeterminedposition such as a wall or a ceiling, an electrical appliance equippedwith a camera module such as an air-conditioner or a refrigerator, andthe like, and may be connected with the home gateway 1010 through awired interface or may be connected with the home gateway 1010 through awireless interface.

The mobile home device 1014 may be a robot cleaner, a mobile toy, or thelike that includes a movement function, and may be connected with thehome gateway 1010 through a wireless interface. At least one of the homedevices 1010 and 1014 includes a function of sensing a movement througha camera or a sensor, and may report a movement sensing result to thehome gateway 1010. Also, at least one of the home devices 1012 and 1014may be equipped with a light such as an LED so as to capture an image ina dark room.

One or more sensing modules 1016 may be installed in a position that isdetermined to be monitored or may be included in the mobile home device1014, and may be connected with the home gateway 1010 through a wired orwireless interface. The sensing module 1016 may be a door sensor thatsenses intrusion through a front door, a magnetic sensor that sensesbreaking into a window, a ultrasonic sensor, a gas sensor that senses aleakage of gas from a gas range or the like, a heat sensor that senses afire, and a movement sensor that senses a movement of an intruder or apet, and may include a signal transmitting module that wiredly orwirelessly transmits an abnormal signal including correspondingidentification information when an abnormal circumstance is sensed whilea home network system operates in a security mode. The abnormal signalmay be collected by the home gateway 1010 or may be directly transferredto the home devices 1012 and 1014 equipped with a camera.

The home gateway 1010 may operate the home network system 1000 in asecurity mode based on a request from a user or a predetermined setting.When the security mode is begun, the home device 1010 and 1014 equippedwith a camera begins capturing image data, and stores the image data orreports the image data to the home gateway 910. The home gateway 1010may store movement commands that instruct the mobile home device 1014 toarrive on at least one target position that requires additionalmonitoring, for example, a position where a sensing module is notinstalled, a position having a high probability of intrusion, a placewhere a child or pet exists, or the like, together with timeinformation. The mobile home device 1014 may be configured to executecapturing image data and a sensing operation by patrolling a pathbetween a docking position and at least one target position when asecurity mode operates. The mobile home device 1014 may store themovement commands by itself

In the security mode, the sensing module 1016 begins a sensingoperation.

In step 1030, the sensing module 1016 senses an abnormal circumstance,and transmits, an abnormal signal including its identificationinformation to the home gateway 1010. As an optional embodiment, whenthe home gateway 1010 is capable of identifying an identity or aposition of the sensing module 1016 through a wired or wirelessconnection, transmission of the identification information may beomitted.

In step 1032, the home gateway 1010 transmits an alarm messageassociated with an abnormal circumstance to at least one registeredmobile terminal 1022 in response to the reception of the abnormalsignal, and the mobile terminal 1022 may display or output a warningmessage in response to the reception of the alarm message from the homegateway 1010. As an optional embodiment, the home gateway 1010determines whether to inform the mobile terminal 1022 of an abnormalcircumstance sensed by the sensing module 1016, based on scenariosregistered in advance. As an embodiment, when a user goes out, the usermay register, in the home gateway 1010, a scenario in which a child or apet is located in a main room. Then, the home gateway 1010 determinesheat or a movement sensed from the main room as a normal circumstance,and determines not to transmit an alarm message to the mobile terminal1022. As another embodiment, when food is being cooked or is scheduledto be cooked in a kitchen, a user may register, in the home gateway1010, a scenario in which heat is generated in a kitchen. Then, the homegateway 1010 determines heat or a movement sensed from the kitchen as anormal circumstance, and determines not to transmit an alarm message tothe mobile terminal 1022. When a circumstance besides the registeredscenario happens, the home gateway 1010 determines to inform the mobileterminal 1022 of an abnormal circumstance. As another embodiment, when acircumstance corresponding to a registered abnormal scenario happens,such as sensing that a front door or a window is open, sensing amovement, and the like, the home gateway 1010 determines to inform themobile terminal 1022 of an abnormal circumstance.

In step 1034, the mobile terminal 1022 executes a remote controlling appthat supports home monitoring, automatically or based on an input of auser (a touch or a voice), in response to the reception of the alarmmessage from the home gateway 1010, so as to connect with the homegateway 1010 through the network 1020, and to transmit, to the homegateway 1010, a request message for requesting home monitoring.

The home gateway 1010 collects image data from at least one home device1012 and 1014 equipped with a camera, in response to the request messagein step 1036, and transmits the collected image data to the mobileterminal 1022 in step 1038. The mobile terminal 1022 displays the imagedata so that a user checks the image data.

As an embodiment, the home gateway 1010 collects image data from the atleast one home device 1012 and 1014 that is registered in advance andlocated in an identical space (for example, a room) where the sensingmodule 1016 that senses an abnormal circumstance exists, and providesthe collected image data to the mobile terminal 1022. As anotherembodiment, the home gateway 1010 collects image data from all of thehome devices 1012 and 1014 equipped with a camera, and provides theimage data to the mobile terminal 1022. As another embodiment, the homegateway 1010 collects the image data from at least one home device 1012and 1014 registered in advance for home monitoring, and provides theimage data to the mobile terminal 1022.

The image data may be transferred in an original resolution or in a formof a compressed small resolution, based on a setting of a remotecontrolling app executed in the mobile terminal 1022. When the homedevice 1012 and 1014 transmits image data to the home gateway 1010 inreal time, and the home gateway 1010 stores the image data, the homegateway 1010 transmits, to the mobile terminal 1022, the image datacollected in real time without a request for transmission of the imagedata. As another embodiment, when the home gateway 1010 receives arequest message for requesting home monitoring from the mobile terminal1022, the home gateway 1010 requests at least one home device 1012 and1014 to transmit image data and receives the image data from the homedevice 1012 and 1014 in response to the request.

In step 1040, the mobile terminal 1022 transmits, to the home gateway1010, a control command for selecting an image of another camera, or foradjusting a position, an angle, a zoom of a predetermined camera. Instep 1042, the home gateway 1010 transfers the control command to a homedevice (for example, the home device 1014) equipped with a selectedcamera. The home device 1014 transfers, to the home gateway 1010, imagedata captured after moving or controlling a camera lens in response tothe control command in step 1044, and the home gateway 1010 transfersthe image data to the mobile terminal 1022 in step 1046. The mobileterminal 1022 displays the image data so that a user checks the imagedata.

When an abnormal signal is received from the sensing module 1016, thehome gateway 1010 may transmit an alarm message to the mobile terminal1022 and may transmit an alarm message to an emergency rescue center1024 in step 1048. The alarm message includes information associatedwith an abnormal circumstance and a location where an abnormalcircumstance occurs (a home address and the like), and may betransmitted to the emergency rescue center 1024 through the network 1020such as the Internet. Contacts (a phone number, a destination IPaddress, and the like) of the emergency rescue center 1024 may be set inadvance in the home gateway 1010 by a user or a manufacturer. Theemergency rescue center 1024 may be, for example, a police station, apolice box, a fire station, a 119 center, a fire station, or a securityoffice or a private security service company of an apartment, abuilding, and the like corresponding to a building complex or acollective housing.

FIG. 11 is a message flowchart illustrating a home monitoring procedureaccording to an embodiment of the present disclosure.

Referring to FIG. 11, in step 1102, a mobile terminal executes a remotecontrolling app including a home monitoring function in response to arequest from a user (a touch input or a voice command) or according toan execution condition set in advance. When the remote controlling appis executed, the remote controlling app connects with a home gateway toexecute a login procedure or determines a login procedure performed inadvance to permit a connection with the home gateway. In step 1104, thehome gateway collects and stores image data from at least one homedevice that includes a camera module and is installed in a home networksystem, in real time or periodically, irrespective of a request for homemonitoring from the mobile terminal.

In step 1106, the mobile terminal transmits a home monitoring requestmessage to the home gateway through the remote controlling app. The homemonitoring request message includes at least one of a login ID andpassword for a connection with the home gateway, an identifier of themobile terminal, and authentication information obtained in advance. Asan optional embodiment, when a request for a section where a userdesires to monitor (room 1/room 2/kitchen/living room and the like) isinput, the home monitoring request message may further includeinformation associated with the section where the user desires tomonitor.

In step 1108, the home gateway determines authority of the mobileterminal in response to the home monitoring request message, and whenthe mobile terminal has an authority level that is capable of receivingimage data for home monitoring, transfers, to the mobile terminal,monitoring information associated with positions to be monitored in ahouse and/or at least one home device including a camera module. Themonitoring information may include positions that may be monitored or isbeing monitored (for example, a room/living room/kitchen and the like)and camera information associated with each installed camera module. Thecamera information includes at least one of a position, a type (forexample, a mobile/stationary type and the like) and a state (forexample, an activated/deactivated state, a state of being charged, amovable state, a wireless signal intensity measured by a home gateway,and the like). The monitoring information may further include athumbnail still image or thumbnail video of image data collected by eachcamera module.

In step 1110, the mobile terminal identifies the monitoring informationthrough the remote controlling app, and selects a place or camera thatis desired to be checked. In step 1112, the mobile terminal transmits,to the home gateway, the home monitoring request message includingindication information that indicates the selected place or camera.

When image data is not reported from a home device associated with aplace or a camera module requested by the indication information, thehome gateway requests the home device corresponding to the indicationinformation to transmit the image data in response to the homemonitoring request message in step 1114. When it is determined that ahome device having a camera module does not exist in the place requestedby the indication information, the home gateway transmits an imagerequest including a command that instructs a home device having a cameramodule and a moveable means, for example, a robot cleaner or a mobiletoy, to move to the requested place and to collect an image. In step1116, the home gateway receives image data from a corresponding homedevice. When the home gateway collects image data from a home device inreal time, step 1114 may be omitted.

In step 1118, the home gateway transmits, to the mobile terminal, imagedata associated with a place or a camera requested by the mobileterminal, and the mobile terminal displays the image data to enable auser to check the image data. As an embodiment, the image data may betransmitted to the mobile terminal, together with a name of acorresponding place, camera information associated with a correspondingcamera module, or identification information of a home device equippedwith a camera module. The mobile terminal may display the image datatogether with the name of the place, camera information, and/oridentification information.

As an optional embodiment, when the mobile terminal transmitsinformation associated with a section (room 1/room 2/kitchen/living roomand the like) for which monitoring is desired, through the homemonitoring request message of step 1106, steps 1108 through 1112 may beomitted, and the home gateway may proceed with step 1118 so as totransmit, to the mobile terminal, image data captured by a camerainstalled in a corresponding section.

While displaying the image data, the mobile terminal may receive aninput of a command for controlling a predetermined camera from a user.The control command includes at least one of adjusting an angle of acamera, zooming in/out, and moving a home device equipped with a camera,and may further include indication information indicating a camera thatis desired to be controlled. As an optional embodiment, the controlcommand may further include a command for controlling an operation of apredetermined home device equipped with a camera. As an embodiment, whenthe home device is a moveable robot cleaner, the mobile terminalreceives, from a user, an input of a command for executing cleaningthrough a voice or manipulating a screen, and generates a controlcommand corresponding to the input command

The mobile terminal transmits, to the home gateway, a control commandinput by a user with respect to a predetermined home device in step1120, and the home gateway transmits the control command to thecorresponding home device in step 1124. In step 1126, the home deviceadjusts an angle of a camera to an upper/lower/left/right side, executeszooming in/out, executes left/right rotation, or executesforward/backward movement, in response to the control command As anembodiment, when the home device is a robot cleaner and the controlcommand instructs to generate an alarm sound, the robot cleaner outputsan alarm sound in response to the control command from the home gateway.Although it will not be illustrated, as an optional embodiment, the homedevice reports, to the home gateway, an execution result (a progress ofexecution, execution being completed, and execution being impossible)associated with the control command, and the home gateway transfers theexecution result to the mobile terminal.

The home device executes an operation associated with the controlcommand, and continuously transmits image data to the home gateway instep 1128, and the home gateway transfers the image data to the mobileterminal so that the mobile terminal displays the image data in step1130. The image data may be transmitted to the mobile terminal, togetherwith camera information associated with a corresponding camera module oridentification information associated with a corresponding home device.

FIG. 12 is a message flowchart illustrating a home monitoring procedureaccording to another embodiment of the present disclosure.

Referring to FIG. 12, in step 1202, at least one of the sensing modulesinstalled in a home network system senses a corresponding abnormalcircumstance, and reports an alarm including identification informationof itself to a home gateway through a wired or wireless interface. Instep 1204, the home gateway determines whether to inform at least oneregistered mobile terminal of the abnormal circumstance, in response tothe alarm report. As an example, the home gateway may determine toinform a mobile terminal of abnormality when the generation of the alarmdoes not belong to a normal scenario registered in advance. When it isdetermined that the alarm belongs to the normal scenario, the homegateway does not inform the mobile terminal of the abnormalcircumstance, and stores a sensed state (a movement, heat, atemperature, and the like) and a sensed position (a window, a room, akitchen, a front door, and the like) associated with the alarm, togetherwith time information. As another example, the home gateway maydetermine to inform a mobile terminal of abnormality when the generationof the alarm belongs to an abnormal scenario registered in advance.

In step 1206, the home gateway may inform at least one registered mobileterminal of an alarm message indicating the abnormal circumstanceassociated with the alarm. As an embodiment, the home gateway mayinform, of the alarm, only at least one mobile terminal that has anaccess right set to allow reception of an alarm from among theregistered mobile terminals. The alarm message of the abnormalcircumstance may include information associated with a sensed state (amovement, heat, a temperature, and the like) and a sensed position (awindow, a room, a kitchen, a front door, and the like) associated withthe alarm.

When the home gateway does not collect and store image data in real timefrom at least one home device that includes a camera module and isinstalled in a house, the home gateway requests at least one home devicein a home network system to start capturing and transmitting image datain response to the report of the alarm in step 1210. As an embodiment,the home gateway may request image data from a home device located in anidentical space where the alarm is generated and/or an adjacent place,based on positional information associated with home devices registeredin advance. As another embodiment, the home gateway requests all of thehome devices equipped with a camera module in the system to collect andreport image data.

When the sensing module that senses the abnormal circumstance in step1202 is installed in a mobile home device including a camera module, thehome gateway instructs the corresponding mobile home device to interruptpatrolling and to continuously collect image data at a correspondingplace. When a home device including a camera module does not exist in aplace/space where abnormality is sensed, the home gateway may instruct amobile home device including a camera module, for example, a robotcleaner or a mobile toy, to move to the place where the abnormality issensed and to collect and report an image, in step 1216. To this end,the home gateway may store movement commands in advance for moving amobile home device from a docking position to predetermined positions.As another embodiment, the movement commands may be stored in a mobilehome device, and the mobile home device may execute the movementcommands based on a previously set rule, when the mobile home device isoperated in a security mode.

In step 1212, the home gateway begins receiving image data from acorresponding home device, and stores the received image data.

As another embodiment, the home gateway may request all of the homedevices including a camera module to capture and report image data, inresponse to the reception of the alarm of step 1202. The home gatewayanalyzes the reported image data, and combines a result of analysis ofthe image data and states of sensing modules so as to determine whetheran abnormal circumstance occurs. As an example, the home gatewaydetermines whether a person exists in a house, and may determine whethera person enters through a normal path based on states of the sensingmodules. When it is determined as an abnormal circumstance, the homegateway may operate based on a determined procedure. As an example, thehome gateway transmits a text/video to at least one registered mobileterminal, and/or transmits an alarm message to a police station or asecurity company.

As another embodiment, when an abnormal circumstance is sensed by asensing module, the home device may immediately capture image data andbegins to report the image data to the home gateway. To this end, thehome device may be configured to enable communication with a sensingmodule, or to include a sensing module.

In step 1208, the mobile terminal executes a predetermined alarmoperation in response to the reception of the alarm message. As anembodiment, the mobile terminal displays, on a screen, a text message,for example, “a movement is sensed in a front door”, and may execute anadditional alarm operation such as flashing an LED, generating an alarmsound, generating a vibration and the like. As another embodiment, themobile terminal may automatically execute a remote controlling app forhome monitoring in response to the reception of the alarm message. Asanother embodiment, the mobile terminal displays a shortcut icon forexecuting a remote controlling app on a screen in response to thereception of the alarm message, and executes the remote controlling appin response to a user input on the shortcut. When the remote controllingapp is executed, the remote controlling app connects with the homegateway to execute a login procedure or determines a login procedureperformed in advance to permit a connection with the home gateway.

In step 1214, the mobile terminal transmits a home monitoring requestmessage to the home gateway through the remote controlling app. The homemonitoring request message includes at least one of a login ID andpassword for a connection with the home gateway, an identifier of themobile terminal, and authentication information obtained in advance. Instep 1216, after authenticating a connection of the mobile terminal,when the home gateway does not collect image data from a home devicelocated in a place where abnormality is sensed, the home gatewayrequests a corresponding home device to transmit image data in responseto the home monitoring request message, and receives the image data fromthe home device in step 1218. When the home gateway collects image datafrom a home device in real time, step 1216 may be omitted.

In step 1220, the home gateway transmits, to the mobile terminal, theimage data collected by the corresponding home device, and the mobileterminal displays the image data to enable a user to check the imagedata. According to an embodiment, the image data may be transmitted tothe mobile terminal, together with camera information associated with acorresponding camera module or identification information of a homedevice equipped with a camera module. The mobile terminal may displaythe image data together with the camera information and/or theidentification information. As another embodiment, when an abnormalcircumstance is sensed in step 1204, the home gateway immediatelyproceeds with step 1220 so as to begin transmission of image data to themobile terminal.

While displaying the image data, the mobile terminal may receive aninput of a command for controlling a camera from a user. The controlcommand instructs at least one of adjusting an angle of a camera,zooming in/out, and moving a home device equipped with a camera, and mayfurther include indication information indicating a camera that isdesired to be controlled.

The mobile terminal transmits, to the home gateway, the control commandinput by the user in step 1222, and the home gateway transmits thecontrol command to the home device in step 1224. In step 1226, the homedevice adjusts an angle of a camera to an upper/lower/left/right side orexecutes zooming in/out, execute left/right rotation, or executesforward/backward movement, in response to the control command. The homedevice executes an operation associated with the control command, andcontinuously transmits image data to the home gateway in step 1228, andthe home gateway transfers the image data to the mobile terminal in step1230 so that the mobile terminal displays the image data. The image datamay be transmitted to the mobile terminal, together with camerainformation associated with a corresponding camera module oridentification information associated with a corresponding home device.

FIG. 13 is a message flowchart illustrating a home monitoring procedureaccording to another embodiment of the present disclosure.

Referring to FIG. 13, in step 1306, a mobile terminal executes a remotecontrolling app including a home monitoring function in response to arequest from a user or according to an execution condition set inadvance. In steps 1302 and 1304, a home gateway collects and storesimage data from at least one home device that includes a camera moduleand is installed in a home network system, in real time or periodically,irrespective of a request for home monitoring from the mobile terminal.Here, a home device #1 equipped with a mobile camera and a home device#2 equipped with a stationary camera are illustrated.

In step 1308, the mobile terminal transmits a home monitoring requestmessage to the home gateway through the remote controlling app. In step1310, the home gateway determines authority of the mobile terminal inresponse to the home monitoring request message, and when the mobileterminal has an authority level that is capable of receiving image datafor home monitoring, transfers, to the mobile terminal, camerainformation associated with at least one home device that includes acamera module and is located in a house.

In step 1312, the mobile terminal identifies the camera informationthrough the remote controlling app, and selects the home device #2equipped with a camera of which an image is desired to be checked. Instep 1314, the mobile terminal transmits, to the home gateway, the homemonitoring request message including indication information thatindicates the selected camera (or the home device #2).

In step 1316, when the home gateway does not collect image data from thehome device #2, the home gateway requests the home device #2 to transmitimage data in response to the home monitoring request message, andreceives image data from the home device #2 in step 1318. When the homegateway collects image data from the home device #2 in real time, step1316 may be omitted.

In step 1320, the home gateway transmits, to the mobile terminal, imagedata collected by the home device #2, and the mobile terminal displaysthe image data to enable a user to check the image data. According to anembodiment, the image data may be transmitted to the mobile terminal,together with camera information associated with a corresponding cameramodule or identification information of the home device #2. The mobileterminal may display the image data together with the camera informationand/or the identification information.

While displaying the image data, the mobile terminal receives a requestfor changing the image data into image data of another camera from theuser in step 1322. As an embodiment, the user inputs, into the mobileterminal, a command for changing from the home device #2 equipped with astationary camera into the home device #1 equipped with a mobile camera.That is, when the user determines that the user has difficulty in fullychecking a desired portion from an image captured by the stationarycamera of the home device #2, the user determines changing a camera soas to capture the desired portion through use of the mobile camera ofthe home device #1. In step 1324, the mobile terminal transmits, to thehome gateway, the home monitoring request message including indicationinformation that indicates another camera to be used (or the home device#1).

In step 1326, when the home gateway does not collect image data from thehome device #1, the home gateway requests the home device #1 to transmitimage data in response to the home monitoring request message, andreceives image data from the home device #2 in step 328. When the homegateway collects image data from the home device #1 in real time, step1326 may be omitted.

In step 1330, the home gateway transmits, to the mobile terminal, imagedata collected by the home device #1, and the mobile terminal displaysthe image data to enable the user to check the image data. According toan embodiment, the image data may be transmitted to the mobile terminal,together with camera information associated with a corresponding cameramodule or identification information of the home device #1. The mobileterminal may display the image data together with the camera informationand/or the identification information.

The mobile terminal transmits, to the home gateway, a control commandfor controlling a camera, which is input by the user while the mobileterminal displays image data, in step 1332, and the home gatewaytransmits the control command to the home device #1 in step 1334. Instep 1336, the home device #1 adjusts an angle of a camera to anupper/lower/left/right side, executes zooming in/out, executesleft/right rotation, or executes forward/backward movement, in responseto the control command The home device #1 executes an operationassociated with the control command, and continuously transmits imagedata to the home gateway in step 1338, and the home gateway transfersthe image data to the mobile terminal in step 1340 so that the mobileterminal displays the image data. The image data may be transmitted tothe mobile terminal, together with camera information associated with acorresponding camera module or identification information associatedwith the home device #1.

FIG. 14 illustrates a User Interface (UI) in a mobile terminal, forexecuting a remote controlling app according to an embodiment of thepresent disclosure. As illustrated in the diagram, the mobile terminalprovides an application/widget icon 1402 for executing an installedremote controlling app, and executes the remote controlling app whensensing a user gesture 1404, for example, a touch, on the icon 1402.

As another embodiment, the mobile terminal provides an icon forinputting a voice command, and executes a remote controlling app so asto execute a corresponding home monitoring function when a user gestureis sensed on the icon or a voice command, for example, “Show me theliving room”, “Let me see my living room”, or the like, is input.

FIG. 15 illustrates a configuration of a screen for home monitoring of aremote controlling app according to an embodiment of the presentdisclosure.

As illustrated in the diagram, when execution of a home monitoringfunction is requested by a user, and an authority level of a mobileterminal that is capable of inquiring about image data captured by acamera is approved by a home gateway, the remote controlling appdisplays thumbnail images 1502 of image data captured by camera modulesinstalled in a home network system in a form of a lattice or a list.Each thumbnail image 1502 may be a thumbnail video captured in real timeby a corresponding camera module, a thumbnail still image extractedbased on the thumbnail video, or a thumbnail still image periodicallycollected from a camera module.

Each thumbnail image 1502 may be displayed, together with a title areaincluding camera information corresponding to a corresponding cameramodule. The camera information includes at least one of a position (forexample, a room/living room/kitchen and the like), a type (for example,a mobile type/stationary type and the like), and a state (for example,an activated/deactivated state/state of being charged/movable state andthe like) of a corresponding camera module. For example, a firstthumbnail image may be displayed together with a title of “room 1” or“room 1, stationary type”, and a second thumbnail image may be displayedtogether with a title of “IP camera 1(living room)” or “IP camera1(living room), mobile type”.

As an optional embodiment, each thumbnail image 1502 may be displayedtogether with a symbol indicating that an abnormal circumstance issensed from a section (rooms, a kitchen, a living room, a front door andthe like) where a corresponding camera module is installed. That is,when an alarm message indicating that an abnormal circumstance is sensedin a predetermined position is received from the home gateway, themobile terminal may display a symbol indicating the abnormalcircumstance (sensing a door open, a movement, heat or the like) byflashing the symbol, together with a title of a corresponding thumbnailimage of a corresponding section. As another embodiment, a thumbnailimage associated with the abnormal circumstance may be displayedtogether with a flashing outline.

As an optional embodiment, only image data captured by a few of aplurality of cameras installed in the home network system may beprovided from the home gateway to the mobile terminal. Then, the mobileterminal may deactivate a thumbnail image of a camera that is notapproved (for example, shading).

FIG. 16 illustrates a configuration of a screen for home monitoring of aremote controlling app according to another embodiment of the presentdisclosure. Although a user interface of a remote controlling appexecuted in a smart phone is illustrated herein, a user interface ofhome monitoring according to an embodiment of the present disclosure mayalso be displayed through various types of electric devices such as aphone, a Tablet PC, a TV, and the like.

Referring to FIG. 16, a remote controlling app displays an arrangementplan 1602 of a house where a home network system is installed, whenexecution of a home monitoring function is requested by a user. Thearrangement plan 1602 displays distinguished sections of an entirehouse, for example, rooms, a living room, a kitchen, a balcony, a frontdoor, and the like, and includes symbols 1604 and 1604 a indicatingcamera modules or home devices equipped with a camera module inpositions where camera modules are installed in each section.

Each symbol 1604 and 1604 a may be a small image similar to an actualshape of a corresponding camera module or home device, or may be a smallimage indicating a type (stationary or mobile type) of a correspondingcamera module. Each symbol 1604 and 1604 a may be represented in a colorimage or a black-and-white image so as to indicate whether acorresponding camera module is activated (or whether the camera moduleis inquirable). When an abnormal circumstance is sensed by a sensingmodule in a predetermined section or an adjacent sensing module, and areport associated with an abnormal circumstance is received from a homegateway to a mobile terminal, the mobile terminal may display theabnormal circumstance by flashing or brightly displaying the symbols1604 or 1604 a in a corresponding position.

The remote controlling app may display additional function keys 1608 forcontrolling a home network system, together with the arrangement plan1602. The function keys 1608 may provide functions, for example,controlling a light, monitoring power consumption, monitoring an image,controlling security, and the like. Also, the remote controlling app mayfurther display a state/control area 1606. The state/control area 1606may selectively provide a display of a state, a display of a control keyand the like for controlling/managing a home network system according toan authority level of the mobile terminal.

When a desired camera is selected by the user in a screen configurationfor home monitoring, the remote controlling app may enlarge and displayan image captured by the selected camera. As an example, when a usergesture is sensed on one of the thumbnail images 1502 of the screen ofFIG. 15, or when a user gesture is sensed on one of the camera symbols1604 and 1604 a in the screen of FIG. 16, the remove controlling appdetermines to enlarge and display an image of a corresponding camera. Asan embodiment, when a user gesture is sensed on a thumbnail image of adeactivated camera, the mobile terminal may not execute any operation ormay display a message indicating that inquiring about an image isimpossible.

As an optional embodiment, when a user gesture is sensed in a sectionwhere a camera does not exist (for example, the section 1604 b of FIG.16), the mobile terminal may transmit, to the home gateway, a messagerequesting an image of a corresponding section. Then, the home gatewayinstructs a home device equipped with a mobile camera module to move toa corresponding section and to record an image. The home gateway tracesa position of the home device and transmits information associated withthe position to the mobile terminal, and the mobile terminal moves acorresponding symbol 1604 along the movement of the home device anddisplays the same.

As an optional embodiment, when a touch and drag is sensed on the symbol1604 corresponding to the home device equipped with a mobile cameramodule, the mobile terminal transfers a movement command associated withthe touch and drag to the home gateway. The home gateway transfers themovement command to the home device, so that the home device movesaccording to the touch and drag.

As an optional embodiment, when a voice command, for example, “show methe living room” or “Let me see my living room”, is input, in a state inwhich a voice function is activated, the mobile terminal may execute theremote controlling app, and may display, on a screen, an image of asection corresponding to the voice command in an enlarged size.

FIG. 17 illustrates a display of image data according to an embodimentof the present disclosure. As illustrated in the diagram, when a touchon a predetermined area or a voice command is input in a remotecontrolling app, a video 1702 that is being captured by a selectedcamera is displayed on a screen of a mobile terminal in an enlargedsize, together with a title area 1704 including camera informationassociated with the video 1702. The camera information includes at leastone of a position (for example, a room/living room/kitchen and thelike), a type (for example, a mobile type/stationary type and the like),and a state (for example, an activated/deactivated state/state of beingcharged/movable state and the like) of a corresponding camera module.

Together with the selected video 1702, a list area 1706 indicating alist of other camera modules may be further displayed. The list area1706 includes a title including camera information corresponding to eachcamera module, and additionally includes a thumbnail image of acorresponding image data.

FIG. 18 illustrates a display of image data according to anotherembodiment of the present disclosure. As illustrated in the diagram, avideo 1802 captured by a controllable camera is displayed in an enlargedsize, together with a title area 1804 including camera informationassociated with the video 1802. The camera information includes at leastone of a position (for example, a room/living room/kitchen and thelike), a type (for example, a mobile type/stationary type and the like),and a state (for example, an activated/deactivated state/state of beingcharged/movable state and the like) of a corresponding camera module.

When a camera module corresponding to the video 1802 is a mobile camera,and an authority level of a mobile terminal that is capable ofcontrolling a mobile camera is approved by a home gateway, a movementcontrol area 1806 through which an input of a movement command for acamera module is received may be displayed together with the video 1802.The movement control area 1806 includes at least one of aforward/backward movement key and a left/right rotation key. Inaddition, together with the video 1802, at least one of a zoom controlarea 1808 and an angle control area 1810 for a corresponding cameramodule may be further displayed. The zoom control area 1808 includeszooming in/output keys of a camera module, and the angle control area1810 includes upper/lower/left/right side keys for adjusting an angle ofa camera module.

When the authority level of the mobile terminal that is capable ofcontrolling a camera is not approved by the home gateway, at least oneof the control areas 1806, 1808, and 1810 may be deactivated (forexample, shading). As an embodiment, when the mobile terminal does nothave authority for moving a mobile camera, the movement control area1806 is displayed to be deactivated, and the mobile terminal may notrespond to an input on the mobile control area 1806.

As an optional embodiment, when a user gesture is sensed in at least oneof the control areas 1806, 1808, and 1810, the mobile terminal transmitsa corresponding control command to the home gateway, and waits for anexecution result (execution being completed or execution beingimpossible) associated with the control command from the home gateway.In this example, a corresponding control area or all control areas 1806,1808, and 1810 may be deactivated during a predetermined waiting timeafter transmission of the control command When the home gateway receivesthe execution result, the mobile terminal activates a correspondingcontrol area so that a user may input a next control command

FIGS. 19A and 19B are screens illustrating a control of a mobile cameraaccording to an embodiment of the present disclosure. As illustrated inthe diagram, a video area 1902 displays a video through a mobile camera,a title area 1904 displays camera information associated with a mobilecamera, and the movement control area 1806 includes at least one of aforward/backward key and a left/right rotation key.

As illustrated in FIG. 19A, when a user gesture 1912 is sensed on one ofthe keys included in the movement control area 1806, for example, aright rotation key, a control command corresponding to the user gesture1912 is transferred to a mobile camera (or a mobile home device equippedwith a camera module) from a mobile terminal through a home gateway.

Referring to FIG. 19B, a mobile camera (or a mobile home device equippedwith a camera module) captures an image that is rotated to the rightside in response to a control command corresponding to the user gesture1912, and image data corresponding to the captured image is displayed onthe video area 1902 of a mobile terminal through a home gateway.

Similarly, when a user gesture 1914 is sensed on one of the keysincluded in the zoom control area 1908, for example, a zoom-in key,image data that is zoomed in may be displayed in the video area 1902 ofthe mobile terminal.

FIG. 20 illustrates an alarm for an abnormal circumstance associatedwith a home network system in a mobile terminal according to anembodiment of the present disclosure.

As illustrated in the diagram, the mobile terminal displays a warningmessage 2002 associated with an alarm message on a screen, whenreceiving the alarm message associated with an abnormal circumstancefrom the home gateway. The warning message 2002 may provide positionalinformation associated with a section where an abnormal circumstanceoccurs, for example, “a suspicious movement is sensed in room 1”. Inaddition, a shortcut key 2004 for executing a remote controlling app maybe displayed, together with the warning message 2002.

When a user gesture is sensed on the shortcut icon 2004, the mobileterminal executes a remote controlling app, and the remote controllingapp provides a screen configuration for home monitoring, for example,the screen of FIG. 15 or FIG. 16. In the screen configuration for homemonitoring, a thumbnail image or a camera symbol associated with anabnormal circumstance may be displayed to be flashing or in adistinguishing color so as to indicate that an abnormal circumstanceoccurs. When the remote controlling app is executed by the shortcut icon2004, the remote controlling app may automatically execute a designatedfunction, for example, displaying image data associated with a sectionwhere an abnormal circumstance occurs, and the like.

FIG. 21 illustrates a configuration of a screen for home monitoring of aremote controlling app according to another embodiment of the presentdisclosure.

As illustrated in the diagram, the remote controlling app displaysmonitoring items 2102 of image data captured by camera modules installedin a home network system, in a form of a lattice or a list. Eachmonitoring item 2102 includes at least one of a name of a correspondingcamera module, a position (for example, a room/living room/kitchen andthe like), a type (for example, a mobile type/stationary type and thelike), and a state (for example, an activated/deactivated state/state ofbeing charged/movable state and the like) of the corresponding cameramodule, in a form of text or a form of a small image 2104.

As an optional embodiment, each monitoring item 2106 may be displayedtogether with a symbol (flashing or the like) indicating that anabnormal circumstance is sensed from a section (a rooms a kitchen, aliving room and the like) where a corresponding camera module isinstalled. As another optional embodiment, a monitoring item 2108 of acamera that fails to obtain an approval for being inquired or thatabnormally operates, may be displayed to be deactivated (for example,shading).

FIG. 22 illustrates a display of an abnormal circumstance associatedwith a home network system in a mobile terminal according to anotherembodiment of the present disclosure.

As illustrated in the diagram, a remote controlling app displays, in anarrangement plan 2202 of a house in which a home network system isinstalled, distinguished sections of an entire house, such as a room, aliving room, a kitchen, a balcony and the like, and displays a section2204 where an abnormal circumstance occurs to be distinguished so that auser identifies the same. As an example, when a sensing sensor of awindow of room 1 senses damage in a window and reports the damage to ahome gateway, the home gateway transmits, to a mobile terminal, an alarmmessage associated with the window of room 1. The remote controlling appof the mobile terminal displays a section 2204 corresponding to room 1in the arrangement plan 2202 to be flashing or in a distinguishingcolor, and additionally displays a warning message such as “a suspiciousmovement is sensed in room 1”, in response to the alarm message.

When a user gesture is sensed in the section 2204 corresponding to room1, the remote controlling app receives, from the home gateway, imagedata associated with one of the cameras installed in room 1, anddisplays the image data on a screen as shown in FIG. 17 or FIG. 18.

The home network system configured as described in the above maystrengthen monitoring of an area where is vulnerable to security in ahouse, using a mobile home device such as a robot cleaner or a mobiletoy. The mobile home device may be configured to include a cameramodule, and to execute a sensing function such as a movement sensingand/or sound sensing.

In a security mode, a mobile home device and stationary sensing modulesin a house begin execution of monitoring. When a movement or a sound issensed through the mobile home device or the stationary sensing module,the home gateway begins collecting image data through cameras (mobileand stationary types) included in a house, and may analyze image datacollected by the cameras and sensed data sensed by sensing modules. Asan example, the home gateway may determine whether a person exists in ahouse based on image data, and may determine whether an abnormalcircumstance occurs based on whether states of sensing modules and homedevices are changed, and the like. When a result of the analysisdetermines that an abnormal circumstance occurs, the home gatewayoperates based on a predetermined procedure. That is, the home gatewaytransmits a text/image to a mobile terminal of a registered user, andreports abnormality to the police or the security company.

When a security mode is executed, the home gateway instructs a mobilehome device to move to at least one target position set in advance so asto execute image monitoring and sensing. To this end, the home gatewayreceives, in advance, an input of information associated with a placewhere a sensing module is not installed or a position having a highprobability of intrusion, and executes patrol monitoring with respect topositions set by a user when the security mode is executed. When sensingan abnormal circumstance, the mobile home device begins recording imagedata at a corresponding position. When an abnormal circumstance issensed by another sensing module, the mobile home device beginsrecording image data and moves to a place where the abnormalcircumstance occurs.

The mobile home device receives, from the home gateway, informationassociated with target positions such as a place where a child or a petis located, moves to a corresponding target position, executesmonitoring, and reports image data (a video or a still image) to thehome gateway. The home gateway may transfer the reported image data to aregistered mobile terminal of a user.

The home gateway stores positional information of target objects to bemonitored, for example, electrical appliances, in a form of a vectorbased on an entrance or a center of a corresponding section. The mobilehome device may store map data associated with sections in a house ormay receive the same from the home gateway, and may move to the targetobjects based on the vector.

The home gateway may extract a movement vector for a movement to adesired target object using a distance from an entrance of each sectionto a wall and an image of a mobile home device. As another embodiment, amobile home device directly calculates a movement vector and reports themovement vector to the home gateway.

FIG. 23 illustrates calculation of a movement vector of a mobile homedevice according to an embodiment of the present disclosure.

Referring to FIG. 23, the reference numeral 2310 illustrates a groundplan of one section in a house, for example, a room, and the referencenumeral 2320 illustrates an image captured at an entrance 2302 of thesection 2310 by a camera of a mobile home device. A target object 2304to be monitored, for example, a gas valve, may be located on a wallfacing the entrance 2302, and an actual distance from the entrance 2302to the wall is b.

As illustrated in the reference numeral 2320, in an image captured bythe mobile home device, a horizontal distance from the entrance 2302 tothe target object 2304 is measured to be a′ and a height from a floor tothe target object 2304 is measured to be c′.The measured values indicatelengths in the captured image and thus, may be converted into an actualdistance using a screen magnification per unit distance m. The screenmagnification per unit distance m is a value determined based on acamera magnification of a mobile home device.

The mobile home device or a home gateway may calculate, based on b, a′,and c′, a movement vector from the entrance 2302 to the target object2304 as shown in Equation 1.

{right arrow over (v)}=a{right arrow over (x)}+b{right arrow over(y)}+c{right arrow over (z)} a=a′÷(m/b) c=c′÷(m/b)   Equation 1

As another embodiment, the movement vector may be calculated as shown inEquation 2.

{right arrow over (v)}=a{right arrow over (x)}+b{right arrow over(y)}+c{right arrow over (z)} a=a′÷ƒ(m, b) c=c′÷ƒ(m, b)   Equation 2

Here, f( )is a function for calculating a ratio of a distance on ascreen to an actual distance, and may be determined based on a type of acamera.

A movement vector calculated as described above indicates 3D positionalinformation associated with the target object 2304 in the section 2310.

When a mobile home device operates in a security mode, or receives aninstruction from a user or the home gateway, the mobile home devicemoves to the target object 2304 based on the movement vector, andcaptures the target object 2304 by adjusting an angle and a zoommagnification of a camera module. The captured image is transmitted tothe home gateway and/or the mobile terminal, so that the user observesthe target object 2304 in detail.

When a user desires to check circumstances in a house, the presentdisclosure operated as described above provides image data through amobile terminal so as to promote user's convenience, and controls amobile camera for a place that is not monitored by a stationary cameraso that the user readily views an image of a desired place. Also, whenan abnormal movement occurs in the house, a user outside the house mayreadily recognize the abnormality through the mobile terminal, andchecks circumstances in the house through image data and thus, dangerouscircumstances that may occur in the house may be prevented.

Although specific exemplary embodiments have been described in thedetailed description of the present disclosure, various modificationsmay be made without departing from the scope of the present disclosure.Therefore, the scope of the present disclosure should not be defined asbeing limited to the embodiments, but should be defined by the appendedclaims and equivalents thereof.

What is claimed is:
 1. A robot cleaner comprising: drive elementsconfigured to allow the robot cleaner to move in an area; at least onecamera; a communication interface; and at least one processor configuredto: obtain a request from a mobile terminal through the communicationinterface, and if the request is associated with a security operation:control the drive elements for the robot cleaner to move along apredetermined path including a docking position and a target position,control the drive elements for the robot cleaner to rotate to apredetermined direction at the target position to obtain at least oneimage for the predetermined direction using the at least one camera,detect movement of an object based on the at least one image obtainedusing the at least one camera, and in response to detecting the movementof the object: obtain two or more images using the at least one camera,and transmit the two or more images to the mobile terminal via thecommunication interface.
 2. The robot cleaner of claim 1, furthercomprising: a memory configured to store information related to asecurity mode, wherein if the request is associated with the securityoperation, the at least one processor is further configured to: controlthe drive elements for the robot cleaner to move along the predeterminedpath including the docking position and the target position based on theinformation related to the security mode, and control the drive elementsfor the robot cleaner to rotate to the predetermined direction at thetarget position to obtain the at least one image for the predetermineddirection using the at least one camera based on the information relatedto the security mode.
 3. The robot cleaner of claim 2, wherein the atleast one processor is further configured to: obtain first informationrelated to the predetermined path and second information related to thepredetermined direction, and store the first information and the secondinformation in the memory, and wherein the information related to thesecurity mode comprises the first information and the secondinformation.
 4. The robot cleaner of claim 3, wherein to obtain thefirst information and the second information, the at least one processoris further configured to obtain at least one of the first informationand the second information from the mobile terminal via thecommunication interface.
 5. The robot cleaner of claim 1, furthercomprising: cleaning elements, wherein the at least one processor isfurther configured to control the cleaning elements for the robotcleaner to perform cleaning if the request is association with acleaning operation.
 6. The robot cleaner of claim 1, wherein to obtainthe at least one image, the at least one processor is further configuredto control the at least one camera to obtain a plurality images whilethe robot cleaner is at the target position.
 7. The robot cleaner ofclaim 1, further comprising: an illumination device, wherein the atleast one processor is further configured to use the illumination deviceto obtain the two or more images.
 8. A method for operating a robotcleaner comprising: obtaining a request from a mobile terminal; and ifthe request is associated with a security operation: moving along apredetermined path including a docking position and a target position,rotating to a predetermined direction at the target position to obtainat least one image for the predetermined direction, detecting movementof an object based on the obtained at least one image, and in responseto detecting the movement of the object, obtaining two or more images,and transmitting the two or more images to the mobile terminal.
 9. Themethod of claim 8, wherein moving along the predetermined path includingthe docking position and the target position comprises moving along thepredetermined path including the docking position and the targetposition based on information related to a security mode, and whereinrotating to the predetermined direction at the target position to obtainthe at least one image for the predetermined direction comprisesrotating to the predetermined direction at the target position to obtainthe at least one image for the predetermined direction based on theinformation related to the security mode.
 10. The method of claim 9,further comprising: obtaining first information related to thepredetermined path and second information related to the predetermineddirection; and storing the first information and the second informationin a memory of the robot cleaner, and wherein the information related tothe security mode comprises the first information and the secondinformation.
 11. The method of claim 10, wherein at least one of thefirst information and the second information are obtained from themobile terminal.
 12. The method of claim 8, further comprisingperforming cleaning using cleaning elements of the robot cleaner, if therequest is association with a cleaning operation.
 13. The method ofclaim 8, further comprising, to obtain the at least one image, obtaininga plurality images while the robot cleaner is at the target position.14. The method of claim 8, wherein the two or more images are obtainedusing an illumination device of the robot cleaner.